Q-Value Based Particle Swarm Optimization for Reinforcement Neuro- Fuzzy System Design

This paper proposes a combination of particle swarm optimization (PSO) and Q-value based safe reinforcement learning scheme for neuro-fuzzy systems (NFS). The proposed Q-value based particle swarm optimization (QPSO) fulfills PSO-based NFS with reinforcement learning; that is, it provides PSO-based NFS an alternative to learn optimal control policies under environments where only weak reinforcement signals are available. The reinforcement learning scheme is designed by Lyapunov principles and enjoys a number of practical benefits, including the ability of maintaining a system's state in a desired operating range and efficient learning. In the QPSO, parameters on a NFS are encoded in a particle evaluated by Q-value. The Q-value cumulates the reward received during a learning trial and is used as the fitness function for PSO evolution. During the trail, one particle is selected from the swarm; meanwhile, a corresponding NFS is built and applied to the environment with an immediate feedback reward. The applicability of QPSO is shown through simulations in single-link and double-link inverted pendulum system. KeywordsNeuro-fuzzy system, particle swarm optimization, reinforcement learning, Q-learning.